Logarithmic Time Online Multiclass prediction

We study the problem of multiclass classification with an extremely large numberof classes (k), with the goal of obtaining train and test time complexity logarith-mic in the number of classes. We develop top-down tree construction approachesfor constructing logarithmic depth trees. On the theoretical front, we formulate anew objective function, which is optimized at each node of the tree and createsdynamic partitions of the data which are both pure (in terms of class labels) andbalanced. We demonstrate that under favorable conditions, we can construct loga-rithmic depth trees that have leaves with low label entropy. However, the objectivefunction at the nodes is challenging to optimize computationally. We address theempirical problem with a new online decision tree construction procedure. Exper-iments demonstrate that this online algorithm quickly achieves improvement intest error compared to more common logarithmic training time approaches, whichmakes it a plausible method in computationally constrained large-k applications.